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ABSTRACT * f 

, A team of evaluators assessed the implementation of 
the Career Education Incentive Act in Tennessee during the period 
from the >'f all of 19,79 through January 1981. Using a series of 
standardized reporting forms as well as telephone and personal 
interviews wi.th local project directors, ^valuators collected data 
pertaining to 20 local projects Carried out in 1979-80 and 17 local 
proj^qts conducted in 1960-81. They found that a majority of the 
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198(T*81 local career education projects accomplished most of the 
-objectives specified in Tenness^e^s state plan for career educati 
. indlud^d among the accomplishments were the following: conduct 'of 
nee,ds assessments to formulate local project objectives, purchase 
instructional and career' guidance materials, development of plans 
reduce bi^s/stereotypxng, establishment of resource cen'ters and 
advisory councils, % and collaboration with community organizations; 
Only in the areas o % t advisory^ouncil meetings and staff awareness 
sessions did achievement of criterion-referenced objectives fall 
short of' predicted levels'. Factors associated with project success 
and failure were isolated. Among factors contributing to project 
failure were absence of & project director, failure *ta conduct needs 
assessments and evaluations, and insuf'f icient/f unding, -Factojrs linked 
to project success included staff development activities, community 
participation^ and early commitment local fuftds # . (MN) 
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Implementation of the Career Education Incentive Act in Tennessee: 
/ ' An Evaluator f s Perspective 

. _ Trudy W. Banta and Judith A/ Boser 

University of Tennessee, Knoxville 

-Introduction * , .... 

Implementation of the State Plan for Career Education in Tennessee 
according, to provisions o£ the Career Education' Incentive Act (PL. 95-207) ; 
began in Fall 1979, - During .the school years 1979-80 and 1980-81, before 
mas sive* budget cuts forced -the elimination of funding for an evaluation 
component,^ personnel jji the Bureau of Educational Research and Service at 
the University of -Tennessee, Knoxville worked under contracts with the 
Tennessee Department^of Education and local, school systems to evaluate the - 
implementation process, r * ^ 

During the first project year t^ evaluators contracted with the State 
\> Coordinator Jf or Career Education to. provide formative evaluation of Tier 

state-wideAeadership activities, iifcliidinf: , 

o 1) administration/coordination procedures 

2) work with the State Advisory Souncil, and j 

3) conduct of state and regio£al*meeting£, i.et 

a) three' technical assisfgnce" workshops; 

b) a state-wide workshop, &n bias'/stereqtyping in career education; 

c) three workshops for counselors; and ' / 

d) a state-wide co^pferBntfe for educators, employees, and represen- 
tatives of public, and^ "private community agencies,"* f> 

1 < In providing leadership and ' coordination for career education activities 
in Tennessee during 197,9.-80 the State Coordinator .attained all except two of 
her seated objectives. By tfhe .end of the project year a listing of consultants 
for local education agencies had not been (completed, .but .this list was 
finalized in January 1981. ^Wh^le preiimindfcy^ contacts had been made, represen- 
tatives of teacher preparation institutions in the State lad not been brought; 
together to. discuss the implications of fcareer education for their programs, * 

During the second project year, 1980-81, the evaluators entered int'o * 
contracts with individual ^chpol systems fo£ evaluation of local projects, 
which received 85 percent of $tate funds 'provided under the terms of the 
Career Education Incentive Act. Methodology for evaluation of local projects 
was designed to 'permit generalizations across projects and thus provide a 
t ?tate-wide evaluation of the career education .program in Tennessee. 

Descriptive Information Concerning Local Projects 

' Early in the first ptoject^year the ^valuators developed a series of 
standard reporting forfcs for directors of local projects, periodic written 
reports, telephone contacts and personal interviews with local directors over 
two years provided the information for bot;h formative and summative evaluation 
of the several projects. -Thfcse data collection procedures provided the follow- 
ing descriptive information about the 20 ioeal projects conducted in 1979-80 
and the 17 local projects 'oarried out in/l980-8ll _ 



•More than 2000 educators in Tennessee received career education 
training through the local projects financed in part vith Career * 
Education Incentive Act funds. /*\ 
* s • • / * . Af 

.Teachers in lo^al projects reported that they hid conducted infusion 
activities in more than 5000 classrooms during;/the two project years. 
If each classrqom contained at least 25 stude$t%,_ some 125,000 
- m students received instruction in career education as a result of 
PL 95-207 in Tennessee. $ 

.More than 10(}' career resource centers were Established in Tennessee 
with Incentive Act fund's. At the end of thl second year project 
directors reported that 'approximately 60 percent of the teachers 
involved in their progtams had visited onef of the centers, and that 
three-fourths of the students taught by these teachers had used . 
materials frpm the denters. t J', m 
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.Almost 1300 students were provided work experiences for the purpose 
of career exploration through the local' project** ^ 

• ' so 

Many of the objectives specified in^Tenpessee's State Plan for Career 
Education pertained to the achievements of local projects. Tftese objectives 
were stated in measurable terms, i.e.", so that the percentage of local 
projects achieving each could be* determined Criteria 'were established which 
specified that the percentage of projects attaining the objective wpuld " * 
increase eacti year. A comparison of the criterion level and percentage, of " 
projects achieving each objective during the second project year (1980-81) % 
is shown in Table. 1. * ' 

„ As, the data in Table 1 illustrate, a majority of* the 1*980-81,. local 
career education projects accomplished the* following: * ^ - ' 

•conducted a needs assessment , to assist in formulation of objectives" 
for the^local project (88% of the -projects X- \ 
• » i . 

♦purchased .instructional (and^ career- guidance. materials (88%) anci 
instructed users in appropriate methods of delivery (71%). 

.conducted ins^rvice training in career education concepts and 
infusion techniques (88%). 

I 

^.included in trheir plans attempts to reduce bias /stereotyping, /( 71%) '. 

- .set up a career resource center to serve both students and £he/ 
public (64%). • , r ;\V ' 

.established an advisory council (88%) which met at least oifca (71%).* 

.identified and used an evaluation instrument (88%). k 1 

.implemented collaborative activities with community organizations, (76%) 

.adopted a statement concerning career education as part of school - . ■ 
board goals/policies '(71%) . * j , . 
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Project % plans include emphasis on A4, 

reducing bias/stereotyping* » 2 

Evidence that at least 3 activities A4„ 

were carried out to reduce bias/stereotyping 1 

• - 1 y 

Provided inservicfe in career education Blj 
concepts, infusion techniques 

School board has adopted, c^areer education as "CI- 

a goal or as* the subject' of a policy statement l t 



Conducted a local needs assessment and. 
identified objectives to meet these needs 

v Established an Advisory Council 

Developed a curriculum guicle in a 
basic skill area 

■ \< • < ^ • ' 

^ Establisfted a comprehensive career 
. guidance program • * * t 

s * 

* - • m 

Jdentified and used an evaluation instrument 
7 

Established career resource center (s) to 
serve students and the public 

'Implemented collaborative activities with 
.Community organizations / 9 



C2 



C2, 



62, 



C2 C 



C2, 



C2- 



C2 8 



Implemented wOrk experiences for the purpose C2 
*of career .exploration . 

Provided awareness sessions for other st^ff , C3« 
to enable them, to assess bias/stereotyping^ 
in materials 
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(Table 1, continued) 
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Purch§sed instructional and career guidance % C3^ 
materials 

Instructed users in appropriate <, • ^ C3^ 
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Advisory Council has met . C3 
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Fewer than half the local projects accomplished the following objectives, 
but the percentage nevertheless exceeded the criterion set out Tin the State 
Plan: m \ 

•developed a curriculum guide in a basic <skill area, \ 

.established a comprehensive career guidance program. 

.provided work experiences £or^he purpose of career exploration. 

In only two areas did achievement of the criterion-referenced objectives 
fall short of predicted (levels. Whereas 75 percent of the local projects 
should have held at least one meeting -of an advisory^council during 1980-81, 
just 71 percent of the projects actually did so. Seventv-one percent of the 
projects described plans to emphasize reduction of bias /stereotyping when 
75 percent should have done this. On^Ly 65 percent 6f the projects provided \ 
awareness sessions for. staff to enable them -to assess bias/stereotyping in 
materials; 90 percent were supposed to do this during ^SO^l.* Finally, 
although 75 percent of the projects should have carried out at least three 
activities n tol reduce bias/stereoty^ig ; only 53 percent did so. 

Progress .Toward" Improving Student Achievement of Career Education Outcomes 

— s v *■ 

Early in the first project. year the evaluators began to emphasize the 
Heed ta assess the quality of the career education program being provided 
for^ students in Tennessee in terms of % (a) meeting 'identified student needs 
and (b) improving student achievement of career education objectives. A 
number of factors combined to make progress toward this end difficult to 
achieve. First, the objectiv.es as stated in Tennessee's State Plan, for 
Career Educatio'n were largdly process-oriented, i.e., they could be evaluated 
simply by asking M Was this done?" without focusing on how well it was done. 
Second, the evaluators had no opportunity during 1979-80 to suggest to local 
project directors ^strategies "for measuring student outcomes. Even during 
1980-81, when consultants were employed to conduct individual evaluations of 
11 of 1^ local projects, the evaluators were not contacted until the projects 
were in their second month of operation. Objectives had been written* 
budgets ha^i been committed, and in.'many cases no plans had been made to 
measure student achievement^ v. 

"Eight of , the 17 local projects funded in 1980-81 provided evideftce of 
haying utilized measures of student outcomes. Four used the Career Maturity 
Inventory.. Of the four, two reported that students who had experienced a 
.career education program made significant gains on one or 'more CMI subscales. 
Students involved in one project improved their pre-test scores on the Goal 
Selection an$i Planning Sqales. A secoftd program was successful in improving 
scores on the Attitude Scale. N 

The Career Orientation Battery" was given to third and fifth graders in* 
one project with the result that 4 third graders involved in career education 
increased their scores on the Sex Equity .scale of the instrument. 

^/In another project^ students who had been in a career education program 
for a year achieved higher scores than a comparison group v on six scales of' 
the Career Skills' Assessment Program instrument prepared by the College Board. 
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Comparison of Project Accomplishments in Year 1 and Year 2 ' • 

• - - ■' • 

Thirteeii projects received continuation funding in 1980-81. Fairly 
complete information was available for 12 of these projects for each of the 
twcj years (see Tables 2 and 3) , Analysis of the reports prepared "by evalu- 
ates and project directors suggests a number of generalizations- Some ' 

• project activities Reared to be more likeLy to receive emphasis during the 
initial. year, while others took longer to effect and -accomplishment was • 
more apparent durirfg thfe second year, * 

Project Management . Five projects employed' part- or full- time* coordin- * 
ators during the first year and only oneVadditional project, pr six altogether, 
had a coordinator during the second year, * ' * 

Needs Assessments . All -projects fpr which information was available 
during the first y^ar (12) conducted needs assessments during* that year, 
Seven conducted needs assessments during the second year. ' * . « 

Three projects included students as sources of inforStffbn in neejis 

• assessments conducted during the first year. While detailed information 

about needs assessments was available for only seven of the projects during * * . 
Year 2, six of the seven used students as a .primary source of data. 

Inservice Participation . The number of educators participating in inservice 
train^ig related to career education decreased from Yekr 1 to Year 2. With' 
information available for 12 .projects each year, there was an overall decrease 
pf 37 percent T * » 

Infusion . The average number of classrooms in which infusion occurred 
rose from H8 per project in the first year to 324 during the second year, 
, representing a 175 percent increase, • % 

, Purchase of Materials . All projects reporting v purchased career education » 
materials during both project years. However, f both the number of materials 
*and -the average number of materials purchased per project declined during the 
second year of implementation* 

fe Curriculum Guides . Curriculum guides were developed in five of the 
projects An 1980-81, whereas only one project developed such materials in 
4979-80. As classroom infusion increased (and, presumably, activities were 
.tried in the classrooms) curriculum guides were more likely to be developed. 

- Bias/Stereotyping Activities , Four projects reported conducting activi- 
ties to reduce bias or stereotyping during Yeat 1, while 11 did so during 
* Year 2, > ' 

Work Experience . While the number of projects/reporting that work experi^ 
ences had been' provided for students for the purple of career exploration 
' decreased' from seven the first year to six, the average number of students * 
served per project increased from 38 to 129 . ■ * 

. Generation of Funds , During Year 1 only one project indicated that funds 
had been received from a source, other than federal* or local government. * During *"* 
'Year 2 three projects utilized outside funds, with a combined total of $101,^7,50< 
This finding was based on information from seven^F the projects for the second 
* year. The one project which obtained outside funding during the first year 
shotted receipt of $800 from community resources^ The. project did not supply 
infbrmation 'for the. second year, so it cannot be determined > whether that 
project maintained i^ncreased^ or decreased the ampunt ( of external funds received. 

Publicity . Much greater public exposure accompanied the sedond year of 
funding, „ There was a ^23. percent increase in the number of newspaper articles 4 
that were submitted to publicize career education in the second Y^/m* c More 
projects "submitted magazine articles. Radio and television presentations by 
personnel connected with several projects were made for the first time during. 
^ the second year of funding, \A total of lei^pfubl^city efforts (including 
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Comparison of First and * Second. Year^, 
Project Activities 
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Activity ♦ • 


Number of 
Projects » 


Number of 

Projects 

Reporting 


Number of 
' Projects 


Number of 

Projects 

Reporting 


. . 




Employed* part- or full- . 
time career education 
* coordinator ■< 1 


5 


12 \ 


6 ' 


12 

* 


» 


* 


• Established' an 
advisory council 


12 


12 ^ 


12 


12 


< 




Conducted needs assessment 
current year 


12r^. 


K* - 


7 


12' 
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Need's assessment 
included student? 


" " 3 


s 11 

* * — - 1 


4 


7 
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Conducted needs assessment 
prior to current year 


6 


12. ' 

S 


• 12 


; 12 \ 

0 
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> School board .has adopted* 
career education as a goal 
or subject of a policy 
statement 


io 4 : 


12 




• 12 
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Has a. System-wide plan 
for career education; 


ii 


12- ^ 


12 

f * 


t 

12 


— V 




" ? 
Established career / 

resource centers / 


u 


12 


11' . 


12 






( -Purchased career / 
% educ^tion-Tttaterial6 


ft 

# 1 o 

1 J 


4 13 / 


12 ** 

ft 


12 
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Provided work experiences 
for career exploration 


7 v *■ 


12 r 


% 

. 6 


12 






* / * 
Conducted evaluation 


12 


12 


12 


12 
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Inq}.uded -teachers 
in evaluation / f * * 


11 • . 


,12. 


11 

« 


12 




• * 


Included students 
in evaluation 

— - — ' Ls 


6 


12 


* 6 


12 
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*Table 2 rejkesents the numbers pf projects reporting that they either had 

or had nay conducted certain activities. The figures in Table "3 are based 

' on more specific information from .follow-up questions which not all projects 
supplied/ \ 
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, . " . Table 3 * 

Comparison of First and Second YeaV 
Project 'Activity Averages;, " 

1979-80 : • .1980-81 



TP 



Activity 


Number 


Proj ects 
Represented 


Average 


Number 


•Projects • . 'Change, in 
Represented Average Averaee 


Career Resource 
% centers established 
and maintained 


. 76 N " 


» 

« ii 


6.9"' 


. 7 ' 8 . 


. 10 ' 

V 


■ - < 

'7.8 '. . +.9 . 


Career education , 
materials ^purchased 


6058 


ii 


551 

* • 
— L 


5060 . 


11 


460 ^.91 " . 


Students partici- 
pating in work 
experiences for 
career exploration 


151 

t 


i t 


o 38 

* 


'7 N 71 


• % 

# 


. • . • ) 

129 . • : +91 

t 


Activities tm ■ 
•reduce bias*/ 
stereotyping 


> 

NR 


> 


> 


50 


11 


\ • ' ' 
4.5 v 


DeVelopetf . < 
curriculum guides 


NR 


1 

* 




23 


5 


• 4.* ^ 


Classrooms in which 
career education^ 
was infused 


1063 


* 

t 

• . 9 


• # 

118 


* 

• 3240 

» 


10 

t 


'. 324 a ' +2 0*6 


creators partici- 
pating in inse'rvice 
related to career 
education 


« 

^232 

-> - 


12 


- 

269. 


2044 
• 


• 

* 12 


; 0 , Y '— r 

' 17o'* -99 
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External Funds 
pbtained , 


$800 


1 


$80f) 


Sim 6S7 sn 


* 


$3*3,885.83 +33", 085. 83 


Publicity ^Jbout % * 
career education: 
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# V 

% 


__ } r - 

Newspaper^ articles 


99 • ' 


12 


8.25 


122 , 


12 


« i 

•10.17 +1.92 


Magazine articles 


6 - 


3 


2 - 


10 '' ' 




.\2*.5 > +.5. 


Radio presentations 




( 




■ * 14 


* 

4 


\.5 ; +3.5 * 


TV presentations 








13" 


3 


4.3 +4.3' 


presentations at 
meetings * , 


NR* 






54 


" 4 ' 7 




3 plus f, A£L" ' in one. pr oj ec t , 

'h * 

' not recorded in 1979-80 


"ALL K-4" in 


another projects 




i 



^NR » figures not reported * „ / » 

*Table. 2 represents the numbers oi projects reporting that they eitheT had or had 
- not conducted certain activities. The figures in Table 3 are based on moife specific 
rmatiQn^from follow-up questions wliich not all projects supplied. 



presen^aj^pns at meeting^) was, recorded for the six projects with coordinators 
for the Career education Wogram, while 53 presentations or submissions were 
made by the six projects that submitted information but did not have a part- 
or full-time coordinator, " \. 

Evaluation . . Evaluation r^orts f or «local projects revealed an increase 
in N the use of research design anXstitistical tests during the second project 
year. Eleyen projects utilized teabtmrs as ^sources of 'evaluation data both * 
years, .while six included -students botKyears.f First year project reports 
noted gains in stuxUnt inf ormatioiKand increases in'student self career , ' 
educational, and economic awareness. In the second year t>he student ifeasutes 
were used in*a manner which made it easier to assess project impact on the 0 
students, i.e., pre- and post-tests, some'with comparison groups, were- employed 
For Year 2, significant gains on' the Attitude* Goal Selection, and Planning 
scales of the Career Maturity Inventory were reported; behavioral. objectives 
were stated and met in some projects; the Career education students scpred , 
40% higher than a comparison group of ' students on various areas of Jthe Career , 
Skills Assessment Program. While there is still ro&m for improvement in 
measurement and evaluation of the projects 1 effects on . s tudejitS, spund evidence 
was produced during the second year of implementation to showS^Ue the projects 
did have a positive impact on students. The need for assistance i&Whe projects 
in the area of evaluation was one which became evident during the fir^k year 
of implementation, and received attention by the State Coordinator. Thi^N 
attention appeared -to hcfve resulted' in improved evaluation procedures. * 

, To'« summarize, the first year of project funding was likely to be , 
characterized by: ' * 1 

.Conducting a needs assessment • 
.Surveying groups other than students in the, needs assessments 
.Providing i^nservice training * v . , 

.Purchasing of materials 

.Conducting f evaluation based on teacher .rather than student outcomes. 

" . - . 4 

.Second year funding wfes more* likely to produce: A 

f ' . 
.Needs' assessment^ in which students were surveyed 

.Fewer projects conducting needs assessments ' ' <t 

jjnffusion.of career education concepts in a- larger number *of classrooms, 

£«*aecrease* in the number of materials purchased ^ 

^Diminis-hed participation in inservice training ^ 
.Development of more curriculum guides • 

.An increase in activities; to reduce' bias dt stereotyping 
.More^ student participation in exploratory work experiences 
.More revenue from external sourcestf or* career education 
^ .More -publicity about career education * > 

^Evaluation of projects characterized by better 'measures, cotter ;~y ' 
design, and better reporting. • X - • / iH^J ' 
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Jactors Involved in Project Continuation 



v In the course of ^analyzing in -various ways the experienced local, 
projects for two year^. the evaluators determined that ther^wHTe several 
factors that mitigated against the sur^val of projects. These included: ' 

/ • . < -/T 

7. the absence of a project director ^ 
.failuVe to conduct^a yearly needs assessment 

.failure to conduct an evaluation that included a measure of ^student outcomes 
^.insufficient funding. 
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Factors that seemed to bode well 'f or 'continuation, even, with little^r 
external fundi'ng ? included: - ~T 

.employing a full-yLme project director 
•.conducting staff development programs 
* .purchasing materials 

.making, career education a componen-t of school board policy 
• establishing ar} advisory council 
•incorporating community participation ^ 
•acquiring early commitjnent c of local funds* 
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